Skip to content

Refine variant reason computation to include SPLIT and STATIC#11037

Open
typotter wants to merge 2 commits intomasterfrom
typo/openfeature-reason
Open

Refine variant reason computation to include SPLIT and STATIC#11037
typotter wants to merge 2 commits intomasterfrom
typo/openfeature-reason

Conversation

@typotter
Copy link
Copy Markdown
Contributor

@typotter typotter commented Apr 2, 2026 •
edited
Loading

What Does This Do

Computes the correct OpenFeature evaluation reason from the UFC allocation structure instead of hardcoding TARGETING_MATCH for all successful evaluations.

Three-way logic in resolveVariant():

  • STATIC — no rules, no shards (unconditional allocation)
  • SPLIT — no rules, shards present (percentage-based traffic split)
  • TARGETING_MATCH — rules present (flag matched a targeting rule)

Also fixes one test assertion that needed updating after rebasing onto #10990: a null targeting key on a static flag now correctly evaluates as STATIC rather than TARGETING_MATCH.

Motivation

System tests Test_FFE_Eval_Metric_Basic (expects reason=static) and Test_FFE_Eval_Reason_Split (expects reason=split) were failing because the evaluator hardcoded TARGETING_MATCH regardless of allocation structure. This aligns Java behavior with the Python and Go SDK reference implementations.

Co-authored-by: Leo Romanovsky leo.romanovsky@datadoghq.com (original fix in #10971)

Additional Notes

The split object is passed into resolveVariant() so the reason can be derived at resolution time without additional lookups.

The semantic question of whether a catch-all (no-rules, no-shards) allocation inside a multi-allocation flag should return STATIC vs another reason is out of scope here — that's a pre-existing behavior question tracked separately.

Contributor Checklist

@typotter typotter marked this pull request as ready for review April 2, 2026 12:33
@typotter typotter requested a review from a team as a code owner April 2, 2026 12:33
@typotter typotter requested review from dd-oleksii and sameerank and removed request for a team April 2, 2026 12:33
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 2, 2026 •
edited
Loading

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@typotter typotter changed the title fix(openfeature): compute correct reason from allocation structure Refine variant reason computation to include SPLIT and STATIC Apr 2, 2026
@typotter typotter requested a review from leoromanovsky April 2, 2026 12:35
sameerank
sameerank approved these changes Apr 2, 2026
View reviewed changes
...e-flagging/feature-flagging-api/src/main/java/datadog/trace/api/openfeature/DDEvaluator.java
Comment on lines +362 to +365
.reason(
!isEmpty(allocation.rules)
? Reason.TARGETING_MATCH.name()
: !isEmpty(split.shards) ? Reason.SPLIT.name() : Reason.STATIC.name())
Copy link
Copy Markdown

@sameerank sameerank Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@dd-oleksii dd-oleksii added type: enhancement Enhancements and improvements comp: openfeature OpenFeature labels Apr 2, 2026
@typotter typotter mentioned this pull request Apr 9, 2026
Draft
@leoromanovsky @typotter
fix(03-01): compute correct reason in DDEvaluator instead of hardcodi…
da5e8a4 
…ng TARGETING_MATCH

- Add Split parameter to resolveVariant for three-way reason computation
- Return STATIC for flags with no rules and no shards (simple flags)
- Return SPLIT for flags with no rules but with shards (shard-based flags)
- Return TARGETING_MATCH for flags with rules (rule-based flags)
- Update 9 test assertions: 7 to STATIC, 2 to SPLIT
@leoromanovsky @typotter
style: apply spotless formatting
63e4fa3
@typotter typotter force-pushed the typo/openfeature-reason branch from 0917bde to 63e4fa3 Compare April 14, 2026 21:17
@pr-commenter
Copy link
Copy Markdown

pr-commenter bot commented Apr 14, 2026

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master typo/openfeature-reason
git_commit_date 1776186597 1776200754
git_commit_sha f064e18 63e4fa3
release_version 1.62.0-SNAPSHOT~f064e18a6c 1.62.0-SNAPSHOT~63e4fa3a40
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1776203322 1776203322
ci_job_id 1595619788 1595619788
ci_pipeline_id 107707909 107707909
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-r2pg1dx0 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-r2pg1dx0 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 10 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:petclinic:appsec:Remote Config better
[-81.613µs; -27.903µs] or [-12.779%; -4.369%]		 583.915µs 638.673µs
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~63e4fa3a40, baseline=1.62.0-SNAPSHOT~f064e18a6c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.06 s) : 0, 1060237
Total [baseline] (11.092 s) : 0, 11091743
Agent [candidate] (1.059 s) : 0, 1059195
Total [candidate] (11.035 s) : 0, 11034506
section appsec
Agent [baseline] (1.252 s) : 0, 1252268
Total [baseline] (11.093 s) : 0, 11092523
Agent [candidate] (1.251 s) : 0, 1251171
Total [candidate] (11.067 s) : 0, 11067041
section iast
Agent [baseline] (1.242 s) : 0, 1241889
Total [baseline] (11.332 s) : 0, 11332180
Agent [candidate] (1.226 s) : 0, 1225751
Total [candidate] (11.233 s) : 0, 11233245
section profiling
Agent [baseline] (1.187 s) : 0, 1187253
Total [baseline] (11.006 s) : 0, 11005768
Agent [candidate] (1.19 s) : 0, 1190377
Total [candidate] (11.132 s) : 0, 11131759
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.06 s -
Agent appsec 1.252 s 192.031 ms (18.1%)
Agent iast 1.242 s 181.652 ms (17.1%)
Agent profiling 1.187 s 127.016 ms (12.0%)
Total tracing 11.092 s -
Total appsec 11.093 s 779.703 µs (0.0%)
Total iast 11.332 s 240.437 ms (2.2%)
Total profiling 11.006 s -85.975 ms (-0.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.059 s -
Agent appsec 1.251 s 191.976 ms (18.1%)
Agent iast 1.226 s 166.556 ms (15.7%)
Agent profiling 1.19 s 131.182 ms (12.4%)
Total tracing 11.035 s -
Total appsec 11.067 s 32.535 ms (0.3%)
Total iast 11.233 s 198.739 ms (1.8%)
Total profiling 11.132 s 97.253 ms (0.9%) 
gantt
    title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~63e4fa3a40, baseline=1.62.0-SNAPSHOT~f064e18a6c

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.246 ms) : 0, 1246
crashtracking [candidate] (1.224 ms) : 0, 1224
BytebuddyAgent [baseline] (633.543 ms) : 0, 633543
BytebuddyAgent [candidate] (632.543 ms) : 0, 632543
AgentMeter [baseline] (29.563 ms) : 0, 29563
AgentMeter [candidate] (29.389 ms) : 0, 29389
GlobalTracer [baseline] (250.136 ms) : 0, 250136
GlobalTracer [candidate] (249.285 ms) : 0, 249285
AppSec [baseline] (32.554 ms) : 0, 32554
AppSec [candidate] (32.273 ms) : 0, 32273
Debugger [baseline] (60.048 ms) : 0, 60048
Debugger [candidate] (59.878 ms) : 0, 59878
Remote Config [baseline] (596.117 µs) : 0, 596
Remote Config [candidate] (591.088 µs) : 0, 591
Telemetry [baseline] (8.097 ms) : 0, 8097
Telemetry [candidate] (8.051 ms) : 0, 8051
Flare Poller [baseline] (8.314 ms) : 0, 8314
Flare Poller [candidate] (9.961 ms) : 0, 9961
section appsec
crashtracking [baseline] (1.228 ms) : 0, 1228
crashtracking [candidate] (1.227 ms) : 0, 1227
BytebuddyAgent [baseline] (663.463 ms) : 0, 663463
BytebuddyAgent [candidate] (663.472 ms) : 0, 663472
AgentMeter [baseline] (12.078 ms) : 0, 12078
AgentMeter [candidate] (12.138 ms) : 0, 12138
GlobalTracer [baseline] (249.877 ms) : 0, 249877
GlobalTracer [candidate] (250.831 ms) : 0, 250831
IAST [baseline] (24.611 ms) : 0, 24611
IAST [candidate] (24.59 ms) : 0, 24590
AppSec [baseline] (185.544 ms) : 0, 185544
AppSec [candidate] (184.48 ms) : 0, 184480
Debugger [baseline] (66.225 ms) : 0, 66225
Debugger [candidate] (65.692 ms) : 0, 65692
Remote Config [baseline] (638.673 µs) : 0, 639
Remote Config [candidate] (583.915 µs) : 0, 584
Telemetry [baseline] (8.505 ms) : 0, 8505
Telemetry [candidate] (8.356 ms) : 0, 8356
Flare Poller [baseline] (3.57 ms) : 0, 3570
Flare Poller [candidate] (3.506 ms) : 0, 3506
section iast
crashtracking [baseline] (1.253 ms) : 0, 1253
crashtracking [candidate] (1.221 ms) : 0, 1221
BytebuddyAgent [baseline] (813.651 ms) : 0, 813651
BytebuddyAgent [candidate] (802.486 ms) : 0, 802486
AgentMeter [baseline] (11.585 ms) : 0, 11585
AgentMeter [candidate] (11.376 ms) : 0, 11376
GlobalTracer [baseline] (242.119 ms) : 0, 242119
GlobalTracer [candidate] (239.671 ms) : 0, 239671
IAST [baseline] (26.226 ms) : 0, 26226
IAST [candidate] (25.797 ms) : 0, 25797
AppSec [baseline] (32.196 ms) : 0, 32196
AppSec [candidate] (31.904 ms) : 0, 31904
Debugger [baseline] (62.407 ms) : 0, 62407
Debugger [candidate] (58.039 ms) : 0, 58039
Remote Config [baseline] (559.007 µs) : 0, 559
Remote Config [candidate] (1.706 ms) : 0, 1706
Telemetry [baseline] (11.726 ms) : 0, 11726
Telemetry [candidate] (14.086 ms) : 0, 14086
Flare Poller [baseline] (3.582 ms) : 0, 3582
Flare Poller [candidate] (3.406 ms) : 0, 3406
section profiling
crashtracking [baseline] (1.172 ms) : 0, 1172
crashtracking [candidate] (1.195 ms) : 0, 1195
BytebuddyAgent [baseline] (692.654 ms) : 0, 692654
BytebuddyAgent [candidate] (695.802 ms) : 0, 695802
AgentMeter [baseline] (9.148 ms) : 0, 9148
AgentMeter [candidate] (9.179 ms) : 0, 9179
GlobalTracer [baseline] (207.98 ms) : 0, 207980
GlobalTracer [candidate] (208.462 ms) : 0, 208462
AppSec [baseline] (32.86 ms) : 0, 32860
AppSec [candidate] (32.804 ms) : 0, 32804
Debugger [baseline] (66.043 ms) : 0, 66043
Debugger [candidate] (65.854 ms) : 0, 65854
Remote Config [baseline] (588.145 µs) : 0, 588
Remote Config [candidate] (574.182 µs) : 0, 574
Telemetry [baseline] (7.857 ms) : 0, 7857
Telemetry [candidate] (7.865 ms) : 0, 7865
Flare Poller [baseline] (3.564 ms) : 0, 3564
Flare Poller [candidate] (3.562 ms) : 0, 3562
ProfilingAgent [baseline] (93.967 ms) : 0, 93967
ProfilingAgent [candidate] (93.482 ms) : 0, 93482
Profiling [baseline] (94.559 ms) : 0, 94559
Profiling [candidate] (94.057 ms) : 0, 94057
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~63e4fa3a40, baseline=1.62.0-SNAPSHOT~f064e18a6c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.055 s) : 0, 1054898
Total [baseline] (8.829 s) : 0, 8828798
Agent [candidate] (1.055 s) : 0, 1054532
Total [candidate] (8.82 s) : 0, 8820273
section iast
Agent [baseline] (1.244 s) : 0, 1244474
Total [baseline] (9.575 s) : 0, 9574936
Agent [candidate] (1.232 s) : 0, 1232026
Total [candidate] (9.551 s) : 0, 9551298
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.055 s -
Agent iast 1.244 s 189.576 ms (18.0%)
Total tracing 8.829 s -
Total iast 9.575 s 746.139 ms (8.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.055 s -
Agent iast 1.232 s 177.494 ms (16.8%)
Total tracing 8.82 s -
Total iast 9.551 s 731.024 ms (8.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~63e4fa3a40, baseline=1.62.0-SNAPSHOT~f064e18a6c

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.22 ms) : 0, 1220
crashtracking [candidate] (1.214 ms) : 0, 1214
BytebuddyAgent [baseline] (632.075 ms) : 0, 632075
BytebuddyAgent [candidate] (631.094 ms) : 0, 631094
AgentMeter [baseline] (29.372 ms) : 0, 29372
AgentMeter [candidate] (29.326 ms) : 0, 29326
GlobalTracer [baseline] (248.626 ms) : 0, 248626
GlobalTracer [candidate] (248.264 ms) : 0, 248264
AppSec [baseline] (32.267 ms) : 0, 32267
AppSec [candidate] (32.153 ms) : 0, 32153
Debugger [baseline] (59.236 ms) : 0, 59236
Debugger [candidate] (58.913 ms) : 0, 58913
Remote Config [baseline] (595.885 µs) : 0, 596
Remote Config [candidate] (583.722 µs) : 0, 584
Telemetry [baseline] (8.027 ms) : 0, 8027
Telemetry [candidate] (8.077 ms) : 0, 8077
Flare Poller [baseline] (7.394 ms) : 0, 7394
Flare Poller [candidate] (8.865 ms) : 0, 8865
section iast
crashtracking [baseline] (1.257 ms) : 0, 1257
crashtracking [candidate] (1.259 ms) : 0, 1259
BytebuddyAgent [baseline] (815.816 ms) : 0, 815816
BytebuddyAgent [candidate] (807.279 ms) : 0, 807279
AgentMeter [baseline] (11.858 ms) : 0, 11858
AgentMeter [candidate] (11.443 ms) : 0, 11443
GlobalTracer [baseline] (242.512 ms) : 0, 242512
GlobalTracer [candidate] (240.863 ms) : 0, 240863
IAST [baseline] (26.252 ms) : 0, 26252
IAST [candidate] (25.961 ms) : 0, 25961
AppSec [baseline] (32.521 ms) : 0, 32521
AppSec [candidate] (29.758 ms) : 0, 29758
Debugger [baseline] (60.396 ms) : 0, 60396
Debugger [candidate] (62.332 ms) : 0, 62332
Remote Config [baseline] (1.725 ms) : 0, 1725
Remote Config [candidate] (529.005 µs) : 0, 529
Telemetry [baseline] (11.686 ms) : 0, 11686
Telemetry [candidate] (12.755 ms) : 0, 12755
Flare Poller [baseline] (3.493 ms) : 0, 3493
Flare Poller [candidate] (3.446 ms) : 0, 3446
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master typo/openfeature-reason
git_commit_date 1776186597 1776200754
git_commit_sha f064e18 63e4fa3
release_version 1.62.0-SNAPSHOT~f064e18a6c 1.62.0-SNAPSHOT~63e4fa3a40
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1776203804 1776203804
ci_job_id 1595619790 1595619790
ci_pipeline_id 107707909 107707909
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-w1jzhn7a 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-w1jzhn7a 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 19 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast:high_load worse
[+72.977µs; +138.036µs] or [+2.888%; +5.463%]		 same
[-12.771µs; +383.522µs] or [-0.172%; +5.160%]		 unstable
[-184.729op/s; +118.541op/s] or [-13.235%; +8.493%]		 2.632ms 7.617ms 1362.656op/s 2.527ms 7.432ms 1395.750op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~63e4fa3a40, baseline=1.62.0-SNAPSHOT~f064e18a6c
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.017 ms) : 17831, 18202
.   : milestone, 18017,
appsec (18.882 ms) : 18692, 19072
.   : milestone, 18882,
code_origins (17.869 ms) : 17698, 18040
.   : milestone, 17869,
iast (17.867 ms) : 17691, 18043
.   : milestone, 17867,
profiling (18.52 ms) : 18336, 18704
.   : milestone, 18520,
tracing (17.849 ms) : 17672, 18027
.   : milestone, 17849,
section candidate
no_agent (18.519 ms) : 18331, 18707
.   : milestone, 18519,
appsec (18.691 ms) : 18504, 18878
.   : milestone, 18691,
code_origins (17.997 ms) : 17822, 18171
.   : milestone, 17997,
iast (18.133 ms) : 17953, 18313
.   : milestone, 18133,
profiling (18.807 ms) : 18621, 18992
.   : milestone, 18807,
tracing (18.026 ms) : 17848, 18205
.   : milestone, 18026,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.017 ms [17.831 ms, 18.202 ms] -
appsec 18.882 ms [18.692 ms, 19.072 ms] 865.351 µs (4.8%)
code_origins 17.869 ms [17.698 ms, 18.04 ms] -147.884 µs (-0.8%)
iast 17.867 ms [17.691 ms, 18.043 ms] -149.899 µs (-0.8%)
profiling 18.52 ms [18.336 ms, 18.704 ms] 503.568 µs (2.8%)
tracing 17.849 ms [17.672 ms, 18.027 ms] -167.328 µs (-0.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.519 ms [18.331 ms, 18.707 ms] -
appsec 18.691 ms [18.504 ms, 18.878 ms] 171.907 µs (0.9%)
code_origins 17.997 ms [17.822 ms, 18.171 ms] -522.34 µs (-2.8%)
iast 18.133 ms [17.953 ms, 18.313 ms] -385.838 µs (-2.1%)
profiling 18.807 ms [18.621 ms, 18.992 ms] 287.74 µs (1.6%)
tracing 18.026 ms [17.848 ms, 18.205 ms] -492.497 µs (-2.7%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~63e4fa3a40, baseline=1.62.0-SNAPSHOT~f064e18a6c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.236 ms) : 1224, 1247
.   : milestone, 1236,
iast (3.279 ms) : 3231, 3328
.   : milestone, 3279,
iast_FULL (6.219 ms) : 6155, 6284
.   : milestone, 6219,
iast_GLOBAL (3.756 ms) : 3699, 3813
.   : milestone, 3756,
profiling (2.149 ms) : 2130, 2169
.   : milestone, 2149,
tracing (1.846 ms) : 1830, 1862
.   : milestone, 1846,
section candidate
no_agent (1.259 ms) : 1246, 1271
.   : milestone, 1259,
iast (3.359 ms) : 3313, 3405
.   : milestone, 3359,
iast_FULL (6.163 ms) : 6101, 6226
.   : milestone, 6163,
iast_GLOBAL (3.675 ms) : 3614, 3737
.   : milestone, 3675,
profiling (2.207 ms) : 2184, 2230
.   : milestone, 2207,
tracing (1.884 ms) : 1869, 1899
.   : milestone, 1884,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.236 ms [1.224 ms, 1.247 ms] -
iast 3.279 ms [3.231 ms, 3.328 ms] 2.043 ms (165.4%)
iast_FULL 6.219 ms [6.155 ms, 6.284 ms] 4.983 ms (403.3%)
iast_GLOBAL 3.756 ms [3.699 ms, 3.813 ms] 2.521 ms (204.0%)
profiling 2.149 ms [2.13 ms, 2.169 ms] 913.634 µs (73.9%)
tracing 1.846 ms [1.83 ms, 1.862 ms] 610.43 µs (49.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.259 ms [1.246 ms, 1.271 ms] -
iast 3.359 ms [3.313 ms, 3.405 ms] 2.101 ms (166.9%)
iast_FULL 6.163 ms [6.101 ms, 6.226 ms] 4.905 ms (389.7%)
iast_GLOBAL 3.675 ms [3.614 ms, 3.737 ms] 2.417 ms (192.0%)
profiling 2.207 ms [2.184 ms, 2.23 ms] 948.591 µs (75.4%)
tracing 1.884 ms [1.869 ms, 1.899 ms] 625.646 µs (49.7%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master typo/openfeature-reason
git_commit_date 1776186597 1776200754
git_commit_sha f064e18 63e4fa3
release_version 1.62.0-SNAPSHOT~f064e18a6c 1.62.0-SNAPSHOT~63e4fa3a40
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1776203644 1776203644
ci_job_id 1595619793 1595619793
ci_pipeline_id 107707909 107707909
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-3fvpgrww 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-3fvpgrww 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:dacapo:tomcat:appsec better
[-1.475ms; -1.125ms] or [-38.308%; -29.200%]		 2.551ms 3.851ms
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~63e4fa3a40, baseline=1.62.0-SNAPSHOT~f064e18a6c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.487 ms) : 1476, 1499
.   : milestone, 1487,
appsec (3.851 ms) : 3628, 4075
.   : milestone, 3851,
iast (2.282 ms) : 2212, 2352
.   : milestone, 2282,
iast_GLOBAL (2.321 ms) : 2251, 2392
.   : milestone, 2321,
profiling (2.097 ms) : 2042, 2152
.   : milestone, 2097,
tracing (2.084 ms) : 2030, 2138
.   : milestone, 2084,
section candidate
no_agent (1.49 ms) : 1478, 1501
.   : milestone, 1490,
appsec (2.551 ms) : 2496, 2607
.   : milestone, 2551,
iast (2.267 ms) : 2197, 2336
.   : milestone, 2267,
iast_GLOBAL (2.332 ms) : 2261, 2402
.   : milestone, 2332,
profiling (2.115 ms) : 2060, 2170
.   : milestone, 2115,
tracing (2.082 ms) : 2028, 2136
.   : milestone, 2082,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.487 ms [1.476 ms, 1.499 ms] -
appsec 3.851 ms [3.628 ms, 4.075 ms] 2.364 ms (159.0%)
iast 2.282 ms [2.212 ms, 2.352 ms] 794.771 µs (53.4%)
iast_GLOBAL 2.321 ms [2.251 ms, 2.392 ms] 834.202 µs (56.1%)
profiling 2.097 ms [2.042 ms, 2.152 ms] 610.192 µs (41.0%)
tracing 2.084 ms [2.03 ms, 2.138 ms] 596.863 µs (40.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.49 ms [1.478 ms, 1.501 ms] -
appsec 2.551 ms [2.496 ms, 2.607 ms] 1.062 ms (71.3%)
iast 2.267 ms [2.197 ms, 2.336 ms] 777.137 µs (52.2%)
iast_GLOBAL 2.332 ms [2.261 ms, 2.402 ms] 841.912 µs (56.5%)
profiling 2.115 ms [2.06 ms, 2.17 ms] 625.192 µs (42.0%)
tracing 2.082 ms [2.028 ms, 2.136 ms] 592.057 µs (39.7%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~63e4fa3a40, baseline=1.62.0-SNAPSHOT~f064e18a6c
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.038 s) : 15038000, 15038000
.   : milestone, 15038000,
appsec (14.896 s) : 14896000, 14896000
.   : milestone, 14896000,
iast (18.288 s) : 18288000, 18288000
.   : milestone, 18288000,
iast_GLOBAL (17.836 s) : 17836000, 17836000
.   : milestone, 17836000,
profiling (14.755 s) : 14755000, 14755000
.   : milestone, 14755000,
tracing (14.902 s) : 14902000, 14902000
.   : milestone, 14902000,
section candidate
no_agent (14.889 s) : 14889000, 14889000
.   : milestone, 14889000,
appsec (14.831 s) : 14831000, 14831000
.   : milestone, 14831000,
iast (18.482 s) : 18482000, 18482000
.   : milestone, 18482000,
iast_GLOBAL (18.009 s) : 18009000, 18009000
.   : milestone, 18009000,
profiling (15.078 s) : 15078000, 15078000
.   : milestone, 15078000,
tracing (14.982 s) : 14982000, 14982000
.   : milestone, 14982000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.038 s [15.038 s, 15.038 s] -
appsec 14.896 s [14.896 s, 14.896 s] -142.0 ms (-0.9%)
iast 18.288 s [18.288 s, 18.288 s] 3.25 s (21.6%)
iast_GLOBAL 17.836 s [17.836 s, 17.836 s] 2.798 s (18.6%)
profiling 14.755 s [14.755 s, 14.755 s] -283.0 ms (-1.9%)
tracing 14.902 s [14.902 s, 14.902 s] -136.0 ms (-0.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.889 s [14.889 s, 14.889 s] -
appsec 14.831 s [14.831 s, 14.831 s] -58.0 ms (-0.4%)
iast 18.482 s [18.482 s, 18.482 s] 3.593 s (24.1%)
iast_GLOBAL 18.009 s [18.009 s, 18.009 s] 3.12 s (21.0%)
profiling 15.078 s [15.078 s, 15.078 s] 189.0 ms (1.3%)
tracing 14.982 s [14.982 s, 14.982 s] 93.0 ms (0.6%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sameerank sameerank sameerank approved these changes

@dd-oleksii dd-oleksii Awaiting requested review from dd-oleksii dd-oleksii is a code owner automatically assigned from DataDog/feature-flagging-and-experimentation-sdk

@leoromanovsky leoromanovsky Awaiting requested review from leoromanovsky

Assignees

No one assigned

Labels

comp: openfeature OpenFeature type: enhancement Enhancements and improvements

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@typotter @sameerank @dd-oleksii @leoromanovsky